REFINED CHERN-SIMONS THEORY AND HILBERT SCHEMES OF 

POINTS ON THE PLANE 



HIRAKU NAKAJIMA 

Dedicated to Professor Igor Frenkel on the occasion of his sixtieth birthday 

Introduction 

Witten realized polynomial invariants of knots and links, such as the Jones polynomial by 
the Chern-Simons theory on S 3 [28]. This construction also gave invariants for three-manifolds 
and links in them. Recently Aganagic and Shakirov propose a refinement of the SU(iV) Chern- 
Simons theory [TJ [2] for links in three manifolds with ^-symmetry, such as torus knots in S 3 . 
As their refinement is based on a deformation of the S and T matrices, let us recall them 
briefly. The S and T matrices give the action of SL(2, Z) on the quantum Hilbert space of the 
theory on T 2 . In the original Chern-Simons theory, the Hilbert space is naturally identified 
with the space of level k integrable highest weight representations of the affine Lie algebra 
corresponding to SU(iV), and S and T come from modular transformations of characters. For 
example, the invariant of the (n,m) torus knot, colored with a representation corresponding 
to a partition A, is given by 

(0|^(iV A -.)^- 1 ,S|0). 

Here K is a matrix corresponding to an element in SL(2,Z) mapping (0, 1) to (n,m), which 
is a product in S, T. And N° 9 is the matrix given by the fusion product of the representation 
for A. The empty partition corresponds to the vacuum representation at level k. The entry 
of the S matrix is the invariant for the Hopf link colored with A and fi. 

In the refined theory, the S, T and N' m are replaced by matrices found by Kirillov [20] and 
Cherednik [5] in connection with Macdonald polynomials. The Hilbert space has the same 
size, but is identified more naturally with the space of Macdonald polynomials of type SU (N) 
with parameters q, t satisfying the relation q N t k = 1. 

The Chern-Simons theory, both original and refined, depends on two positive integers rank 
N and level k. But it is expected to give a Laurent polynomial in y/q, \/t and t N ^ 2 for a link 
in S 3 . This three variable polynomial is specialized to the HOMFLY polynomial at q = t, 
and is conjectured to be the same as the Poincare polynomial of the link homology group 
defined by Khovanov and Rozansky [TBI [19] (see also [17] ) . The existence of the three variable 
polynomial, or the 'large N limit' in the physics literature, is true in the level of S, T matrices. 
They coincide with ones found earlier in the context of refined topological string theory, or 
the motivic Donaldson-Thomas invariants, of the resolved conifold 0(—l) © 0(— 1) — > P 1 by 
Gukov et al. [TO], up to the change of the base on the quantum Hilbert space [To] . 

In this paper we relate the large N limit of the S matrix to the Hilbert schemes of points 
on the affine plane C 2 . This relation is natural from various points of view. At first by the 
work of Haiman [12] the Macdonald polynomials correspond to elements in the equivariant 
.fT-group of the Hilbert schemes with respect to the C* x C*-action given by C* x C* fixed 
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points. The integrality and positivity, which one should expect from the conjectural relation 
to link homology groups, are explained best in this context. 

Secondly the refined topological string theory is partly motivated by Nekrasov's partition 
function for the 4-dimensional gauge theory on M 4 . See [3j [16] for example. For the resolved 
conifold, the relevant gauge theory is the [/(l)-theory, in other words, the Hilbert schemes of 
points. 

Thirdly there is a conjecture on the homology group of the link of a plane curve singularity 
in terms of the virtual Poincare polynomials of the Hilbert schemes of points on the singularity 
by Oblomkov, Rasmussen and Shende [26] (see also [8]). For torus knots this theory is related 
to the Hilbert schemes of points on C 2 via the Cherednik algebra [26l Conjectures 7,8]. 

Finally the equivariant .fT-group of the Hilbert scheme is naturally identified with equivariant 
if-group of C 2n with respect to the S n x C* x C*-action. This is purely combinatorial setting, 
and resembles to the setting in [T7] . 

Our main result (Theorem 14.81) express the large N limit of the 5* matrix in terms of certain 
Euler characteristics of tensor products of two copies of universal sheaves over Hilbert schemes, 
after Schur functors are applied. The derivation is more or less straightforward from the explicit 
form in [TJ and the relation between Macdonald polynomials and Hilbert schemes [12] except 
that we use a version of Cherednik-Macdonald-Mehta identity by Garsia, Haiman and Tesler 
[7]. We remark that we still identify the quantum Hilbert space with the equivariant K group, 
but change the fixed point base to another base given basically by Procesi bundles. 

As an application, we find an explicit formula for the Euler characteristics of a single uni- 
versal sheaf, applied arbitrary Schur functor (Corollary I4.5p . This includes a formula of Scala 
for the case of exterior powers, but general cases seem to be new to the best of the author's 
knowledge. 

Much more still remain to be done. One of the most important questions is how to under- 
stand products of S, T matrices in this framework. We see that the refined Chern-Simons 
theory gives a function in q, t for a fixed N and varying k. But we do not see what happens 
when we change also N. See Remarks 13.91 After this will be understood, we next compute 
products, for example, to check [261 Conjectures 7,8], where the answer is given by a single 
cohomology group, not products of several. The latter question is probably harder than the 
former one. 

The paper is organized as follows. The first section is preliminaries for modified Macdonald 
polynomials and Hilbert schemes. Modified Macdonald polynomials are obtained from integral 
forms of original Macdonald polynomials by applying the plethystic substitution. These are 
naturally connected with Hilbert schemes in Haiman's theory. In Section 2 we study special- 
ization of Macdonald polynomials at q k t N = 1. In Section 3 we review the S and T matrices 
for the refined theory and rewrite them in terms of modified Macdonald polynomials. In Sec- 
tion 4 we use preliminary results in Section 1 to the formula in Section 3 to get a geometric 
interpretation of the large N limit of the S'-matrix. In two appendices we collect notation on 
partitions and symmetric polynomials. 



Acknowledgments. A preliminary version of this paper was written while the author was 
enjoying the hospitality of the Institute for Advanced Study in 2007. At that time the papers 
[H|2] did not appear yet, and the motivation was to understand [10]. The author thanks Sergei 
Gukov for discussion at that time. He also thanks Mina Aganagic for answering his questions 
on [I]. 
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1. Preliminaries 

l(i). Plethystic substitution. In the main body of the paper we will need a N th root of 
q and the square root of t, but we write the function field Q(q,i) instead of Q(q 1 ^ N , y/i) for 
brevity. 

Let R = Q(q,t)[[xi,X2, ■ ■ ■]] be the formal power series ring in variables xi, X2, ...with 
coefficients in Q(q,t). 

We define a A-ring structure on R by 

^k(q) = q k , fait) = t k , ip k (xi) = x k , 

where ijj k is the k th Adams operation, which is a ring homomorphism of R. 

Let A be the ring of symmetric functions. We have a plethysm o: A x R — > R such that 
(— o A) : A R is a ring homomorphism for any A G R and (p k —)'■ R — > R is the Adams 
operation ip k . Given A £ R and a symmetric polynomial /, we denote / o A by f[A]. It is 
called plethystic substitution of A into /. In practice, the plethysm is determined by p k o as it 
is a ring homomorphism in the first variable. See [TTl Chapter 1]. 

For example, if X — x± + x 2 + ■ ■ ■ , then f[X] — f(x±, x%, . . . ), where / in the left hand side is 
considered as an element of A, and the right hand is considered as an element of R. We often 
denote the right hand side also by /, without specifying either / is in A, R or Q(q, t) (g) A. We 
use this capital letter X throughout the paper. If yi, y 2 , . . . are another set of variables, we use 
Y for yi + ?/2 + • • • ■ Various operations are compactly expressed by plethystic substitutions. 
For example, if / has degree n, then f[qX] = q n f. 

More generally the plethysm is defined for any A-ring R in the same way. Later we will use 
the plethysm when R is the Grothendieck ring K(Y) of vector bundles over a variety Y. The 
A-ring structure is given by tensor products. For example, the operation corresponding to the 
k th elementary symmetric function is the exterior product f\ h . 

Let us return back to our R = Q(q, t)[[x±, x 2 , . . . ]] and compute f[—X]. We have 

Pk [-x] = M-x) = -MX) = -p k . 

This is different from Pk{— X) = (— l) k pk(X). Therefore if we want to replace variables by 
their negatives in plethystic brackets, we use the e-symbol, i.e., 

p k [eX] = {-l) k p k . 

Let oj be the ring involution of A sending e n to h n , or equivalently p k to (— \) k ~ l p k - Then 
Pk[— X] = —p k = (—l) k up k = ujp k [eX]. Therefore (oof)[X] = f[—eX] for a symmetric polyno- 
mial /. 

We also use f[X/(l — q)] later. We have 

r A _ X\ + X\ + ■ ■ ■ p k 
P k U \ ~ 1 k ~ 1 h ' 

1 — q 1 — q K 1 — q K 

It has inverse g \-t g[X(l — q)]. In fact, p k [X(l — q)] — (x k + x\ + . . . )(1 — q k ) = p k (l — q k ). 
Let 



i 1 \k=l / k=0 

Then 

1 — t °° / 1 1 — t k \ 

mr—} = pexp {- kT ^kP^)Pk(y)) 
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This is the kernel of the inner product ( , ) g>t in [211 VI] given by (f,g) qt t = if, g[jziX]). 
We also consider 

(°° f ~\ \k—i t \ \ 00 
k=l J k=0 i 



The following expression appears quite often: 
'1.1) ^ r L - = ff(l - ug»W«). 



sGA 



See £fX]for the definition of B x (q,t) and other notations related to partitions. 

l(ii). Modified Macdonald polynomials. Let P x (x; q,t) be the Macdonald polynomial for 
a partition A as in [2TI VI]. Its integral form is defined as 

J x (x; q, t) d = c x (q, t)P x (x; q, t), 

where c x (q,t) is as in $K\ 

Following works of Garsia-Haiman and their collaborators, we introduce the modified Mac- 
donald polynomial by 



- t- 

This subsection is devoted for the review of various properties of Hn(x;q,t) together with an 
introduction of a fundamental operator V. The reference [H] is a nice survey article on this 
topic and other things, which will be recalled later in this section. 
The Kostka-Macdonald coefficient K Xfi (q,t) is defined as 

(1.2) H ll {x;q,t) = J2^{q,t)s x . 

x 

It is related to one in [2U VI (8.11)] by K Xpi (q, t) = t nM K Xfl (q, r 1 ), as S x (x; t) = s x [(l - t)X]. 
Macdonald positivity conjecture says K Xfl (q,t) e Z>o[?, t], which was proved by Haiman [T2] . 
See the next subsection for a little more detail. 

The modified Macdonald polynomials are characterized by the properties 

(1) H„(x;q,t)eQ(q,t){s x [X/(l-q)] | A > //}, 

(2) H^x;q,t)eQ(q,t){s x [X/(l-t)] | A > //}, 

(3) H lt [l;q,t] = \. 
See [HI 3.5.2]. 

We have P^x; q~\ t~ l ) = P^{x;q,t) [21j VI(4.14)(iv)]. It implies 

(1.3) H^x-q,t)=uH^x-q-\t- 1 )q n ^H n M. 

(See [HI 3.5.12].) Let us give a proof for the sake of the reader. We start with 

P»[ T ^-,q,t- 1 ] = P,[ r ^;q-\t] = P»[^q- 1 ,t}. 



Therefore 



H,(x;q,t) =t^ C ^^H,[-tX-q-\t^} = t^ C J^^uH,[etX; q~\t 
cM V) c^q \t) 
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Now the assertion follows from 

c M (g -1 ,*) 



(1.4) 



-n(/i*) + n(/i) 



fW(-t)l 



s£(J 



and -f/^eiV; g 1 , t 



i +-n 



g 1 ,i Other formulas appearing below can be also 



derived from the corresponding formulas for the original Macdonald polynomials. We leave 
the proofs as exercises for the reader hereafter. 

Let ojqj be the algebra endomorphism of Q(q, t) (g> A defined by 



Uq,tPk 



l - t k 



Pk- 



We have 



It is known (2H VI, (5.1)] that 



H fl t(x;q,t) = H^(x;t,q). 



This is rewritten as 
(1.5) 
(See O 3.5.11].) 

The Cauchy formula for the usual Macdonald polynomials [211 VI(4.13) and (6.19)] is 
n[XY^] = ^ t P At (V;g,t)Q M (V;g,t). It is equivalent to 



(1.6) n 

We apply uj to get 

(1.7) n 



XY 



(l-g)(l-t) 



L(i J 



E 



E 



g -n(^) t -n M ^ (X . g;t) ^ (y . g;t) 

^(g-VHfot- 1 ) 

q-^H-^uH^X; q, t)H^(Y; q, t) 



c„(q-\t)^(q,t-i) 

We define another inner product by 
(1.8) (f, g), = (f, ug[(l -q)(l- t)X}) = (-t) dc ^ (f, g[(l - q)(l - t'^X}), 

where degg denotes the degree of g. We have a relation 



;i.9) 



(f,g) q ,t = (f [j^-j) >^[j3|])* 



'-tr dcS9 (f[^l9[r^T])*- 



l -r " L i -t- 

The Cauchy formula f ll.6p is equivalent to 

(1-10) (Hx,H,)* = h»q n{ ^ n ^c,{q-\ty^r l ). 

See [TU (73)]. This can be directly derived from the formula for inner products of Macdon- 
ald polynomials ([211 VI(6.19)]): (Px,P») q ,t = Sx/^/c^t), and the property (f,g) q , t -i = 
(f[-tX],g) q , t . 

We define an operator V (see [HI 3.5.5]) on A® Q(q,t) by 
(1.11) 



\7H = q n ^h n MH 
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This operator can be generalized as follows. Let / be a symmetric polynomial. We define 
A f by 

A f H, = f[B,(q,t)]H,. 
We have V = A £n on the degree n part. 

l(iii). Hilbert schemes of points on the plane and their torus fixed points. We fix 

terminology on Hilbert schemes, in particular, recall weights of tangent and tautological vector 
bundles at fixed points of a torus action. Our convention for the torus action is the same as 
in [21] (see especially [2~4"j Remark 4.4] for the action on functions) instead of [251 E]> wnere t\, 
t 2 are replaced by their inverse. See also [22]. 

Set X = C 2 and let denote the Hilbert scheme of n points in X. Set-theoretically 
it is nothing but the space of ideals I in C[x, y] such that dimC[x, y]/I = n. We have the 
Hilbert-Chow morphism it: — > S n X, where S n X is the n th symmetric power of X. 

We have a C* x C*-action on X and the induced action on X^ n \ To make compatible with 
[T2] we make it 

(x,y) i y (tx,qy). 

So t\, t 2 in [24] are t, q here. 

We consider the equivariant .fT-group Kc* x c*(X^) of ^' n ' with respect to the C* x tr- 
action. It is the Grothendieck group of the abelian category of C* x C*-equivariant vector 
bundles over X^. As X^ is smooth, it is naturally isomorphic also to the Grothendieck 
group of the category of equivariant sheaves. It is a module over the representation ring 
R(C* x C) = Z[q ±l ,t ±l }. 

Tensor products of equivariant vector bundles are well-defined on K c * xC *(X^). It is 
denoted also by <g). 

Later we will use the derived category D^, xCt ,(CohX^) of equivariant coherent sheaves on 
j' n L The equivariant i^-group Kc* x c*(X^) is isomorphic to the free abelian group generated 
by the objects of D^ xCt (CohX^) modulo relation given by distinguished triangles. 

The C* x C*-fixed points in are monomial ideals in C[x, y], and hence are parametrized 
by partitions fx of n. Let J M E X™ be the fixed point corresponding to fx. Then the C* x C*- 
character of the alternating sum of exterior powers of the cotangent space Tf is given 
by 

(1.12) ch A_i77 M X [n] = ]J (1 - t- l ^ s) q a ^ s)+1 ) (1 - ^W+ig-^M) = c ^ q -\ ty^q, r 1 ). 

Note that this expression appears in ( II. 7p . 

Over the Hilbert scheme X'"' we have the tautological bundle whose fiber at I is given 

by 

Of = C[x,y]/I. 

The fiber over the fixed point 1^ is a C* x C*-module whose character is 

Sg/l 

The generating function of the exterior powers of the tautological bundle at the point 1^ 
satisfies 
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Let us consider its determinant line bundle of O^: 

£ = detC [n] = /\ n O [n] . 

We have 

(1.14) ch£ Ifl = JJg°W w = q n b% n M. 

Let £ be the universal (rank 1 torsion free) sheaf over X x X' n l We consider it as a i^-group 
class and denote it by the same letter also. Let £(o,r„) be its fiber at (0, J M ). By [25], (4.1)] its 
character is given by 

(1.15) ch5 (0 ,7 M ) = 1 - (1 - g)(l - i)5 M (<z,t) = 
Since chC[x, y] = 1/(1 — q)(l —t), we have 

chOf 1 

Ch ^ = 1 "chC[^J- 

We later use slant products of £ with classes on C 2 , for example £/[0], £/[C 2 ]. They are 
classes in the equivariant i^-group, defined as 

£/[o] = <tu(e ® ql[o}), ^/P 2 ] = <t*(£) = (1 _ q L_ t f /M, 

where [0] is the class in K c * xC *(X) of the origin, and qi, q 2 are the projections of XxX™ to the 
first and second factors. Note that £/[C 2 } is a localized class in K c * xC *(X) ®b(c*xc*) Q(<Z>^)> 
not in Kc*xC*(X). It is because q 2 is not proper, and the push-forward homomorphism g 2 * is 
defined by formally using Atiyah-Bott-Lefschetz fixed point formula as above. More examples 
of pushforward homomorphisms for non proper morphisms appear later. 

l(iv). Haiman's works. Much deeper connection between Hilbert schemes X^ and sym- 
metric polynomials are known through Haiman's works [121 US]- Let us briefly recall the main 
result. 

Let X n be the reduced fiber product 

X n — f -+ x n = c 2n 

(1.16) p 

IN > S n X = C 2n /S n . 

Then one of main results in [12] says that j'"' is isomorphic to the Hilbert schemes of S n - 
invariant ideals in C 2n so that X n is identified with its universal family. 

As an application of this result together with the result by Bridgeland-King-Reid [I], Haiman 
proved 

Theorem 1.17 ([13]). The functor 

$ d ^ R/„ o p* : D b (CohXW) -)• D b Sn (CohX n ), 

is an equivalence of categories, where D b (C6hX^) is the derived category of coherent sheaves 
on X^ and D b s (Goh X n ) is the derived category of ' S n - equivariant coherent sheaves on X n . 
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The latter derived category is identified with the derived category of bounded complexes of 
finitely generated S^-equivariant C[x, yf )n = C[xi, y\, . . . , x n , y n ]-modules. 

The above equivalence holds also for equivariant derived categories with respect to the 
C* x C*-action. 

Let V = p*Ox n - This is a vector bundle of rank n\ = \S n \ endowed with fiberwise >S n -action. 
We call it the Procesi bundle. Then the functor $ is identfied with 

$ = Kr X [n](V <g> -). 
Taking equivariant Grothendieck group, we have a natural isomorphism 

$: K c * xC *(X [n] ) ^ K SnX c*xC*(X n )- 

We further compose a homomorphism given by the push-forward homomorphism for X n — > 
pt. Since this morphism is not a proper, it is not well-defined in the level of the ordinary 
Grothendieck group. However, it is well-defined if we take the localized Grothendieck group 
-^5„xc*xc*(pt) G^zfe* 1 ,** 1 ] Q(q,t). This can be done by using Atiyah-Bott-Lefschetz formula 
as there is only finitely many (in fact only one) torus fixed point in X n as in the previous 
subsection. It can be also checked by using a standard argument on Hilbert series. See [T31 
§3] for more detail. Recall that the Grothendieck group R(S n ) of representations of S n is 
identified with the degree n part of A via the Frobenius characteristic map. See $B] Therefore 
we combine the above Hilbert series map with Frobenius characteristic map, we get 

The composited map Kc»xC*{^ ) ~^ Q(?>0 ® A- is also denoted by From its construction, 
we can identify the image of 

Corollary 1.18 f [14[ Prop. 5.4.6]). The functor $ induces an isomorphism 

# c * xC *(XN) c± {/ e Q(q,t) <g> A™ | f[(l-q)(l-t)X] G Z[q ±l , t* 1 ] <8> A"}. 

The subgroup of classes supported on the punctual Hilbert scheme 7r _1 (0) is isomorphic to 
Zfa* 1 ,** 1 ] <g>A n . 

Let 1^ be the torus fixed point as in the previous subsection. Then $(/ M ) is the fiber of V 
at I p. Then one of the main result in [12] says that this is equal to H^. 

We decompose V = ® A V x <S> V\ where V x is the irreducible representation of S n corre- 
sponding to the partition A. Then the fiber of V\ at J M , as a C* x C*-module, has character 
K\n(q,t). This proves the positivity conjecture of K\^, mentioned in the previous subsection. 

There is a natural Q(q, t)-valued symmetric bilinear form ( , ) on Kc* x c*(X^) induced 
from the composite of the functor 

RF xW (-® L -) 

and the character ch: R(C* x C*) — > Zfg^t* 1 ]. The Koszul resolution gives us the formula 
(Ip,I v ) = t^ch/^T^XH Comparing flTP2j) with (QUI) , we find that ( , )„ has an extra 

factor q n ^*H n ^>. This is given by the operator V. Therefore 
(1.19) (-,-) = (V- 1 $(-), $(-)),. 

See [HI Cor. 5.4.7]. 
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The operator V itself has a geometric interpretation. We have an endofunctor on D b (Coh X^] 
given by — <g> C, where C is the determinant of the tautological bundle as in the previous sub- 
section. The character of its fiber at J„ is given by g" (mV^. This means that 

$(-<gi£) = V$(-). 

See [H Cor. 5.4.9]. 

Similarly the operator A/ corresponds to the operator — cg> f(0^), where f(0^) is the 
plethystic substitution of the i^-theory class of to /. 
We have 

X 



9)(1"*) 



See [IU (97)]. 

Remark 1.20. There is also a natural inner product on Ks n xc*xc*{X n ) defined in the same 
way as for ( , ). But the equivalence <E> does not respect inner products. In fact, the natural 
inner product on Kg nX c* x c*(X n ) is ( , )*. This follows from 

X 



A 



(i-t)(i-g)J 

and the observation that s\[X/(l — t)(l — q)] corresponds to V x ® Ox n and s M to the tensor 
product of and the skyscraper sheaf at the origin. The discrepancy of inner products comes 
from that $ satisfies 



Hom D fc n(CohA - n) ($(a),$(6)) = Hom flt(CoWW) (a, b), 

as it is a categorical equivalence. We note RHom D i,( CohI [„])(o,&) = HT X [n](a* cg> b), and a 
similar identity for Hom D & (Cohx n )> where a* is the Grothendieck-Verdier duality. Then we 
use VI =V x t®C (see [H "(96)]) to see that (-, -) = (V -1 (-)> -)*. 

l(v). Classes supported on a lagrangian subvariety. We need a variant of Corollary ll.181 
later. Let l)$ be the inverse image of the symmetric product of the x-axis S n ({y = 0}) under 
7r. We also introduce a similar variety Ly exchanging the role of x and y. These are known to 
be lagrangian with respect the natural symplectic form on induced from that on X. (See 
[22J §9.3].) 

Then Corollary 11.181 in this setting yields 

(i.2i) K c *xc*(4 n] ) = {/ e Q(q,t) ® A" I f[(i-t)x] g zfo* 1 ,** 1 ] ® A"}. 

In particular, we have an integral base of K c * xC * (L^) given by 

{s-^x/Ci -*)])}*• 

Classes $ _1 (sa[X/(1 — £)]) appeared in Haiman's work. We briefly recall the result. See [HI 
§5.4.2] for more detail. 

Recall the commutative diagram (11. 16[> . Let V(y) C X n be the zero locus y\ — yi — --- — 
y n = 0, where yi is the pull-back of the coordinate function on X n under /. It is a complete 
intersection in X n and p maps V(y) to Li"'. Then 

^-\s x [X/(l - t)}) = £*® Eom Sn (V xt ,p*O v{y) ), 

where V xt is the irreducible representation of S n corresponding to the transpose partition A* 
of A, and p*CV( y ) has an induced S^- act ion from that on X n . 
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l(vi). Geometric interpretation of the Cauchy formula. Let us give a few applications 
of the Cauchy formula for the Macdonald polynomials under the above identifications of var- 
ious expressions in terms of Hilbert schemes. These calculation will be prototypes of our 
interpretation of refined Hopf link HOMFLY invariants as Hilbert series on Hilbert schemes. 
We take degree n in ( 11. 7ft and substitute X — Y — 1. Then we get 

(1.22) MV(1 -?)(!-*)]= E 



\n\=n 



as uHpfcqJ] = t n ^q n ^ by (Oj) . By (fTTT2|) the right hand side is (ch A-i^JfM)- 1 . By 
the Atiyah-Bott-Lefschetz fixed point formula, we have 

y — A l - —. . = V(-i) d chif d (x["],o x[n] ). 

(See [13j §3] why this is true even though is noncompact.) We safely confuse the derived 
functor with the corresponding homomorphism on the if-theory, and we denote the right hand 
side by 

chRr xW (C xW ) 

hereafter. 

On the other hand, the left hand side of (1 1 . 2 2 [) is the character of S n C[x, y], the n th symmetric 
power of C[x, y\. It is equal to the character of the coordinate ring of S n X. Therefore (11.221) 
implies 

(1.23) chT SnX (O s «x) = chKT xM (O xln] ). 

In fact, this equality is a simple consequence of the fact R,ir*0 X [ n ] = Os^x, which follows as 
S n X only has a rational singularity. 

Note also that this geometric argument gives a stronger assertion that 

S n C[x,y] if cZ = 0, 
otherwise. 



H d {XW,O xln] 



This (I1.23P can be generalized as follows. Let us put X = 1, Y = 1 — u in (jl.7p . The left 
hand side is 



h 



n—k 



k=0 



l-q)(l-t) 

n 

J2(-u) k ch(S n - k Cix,y}®f\ k Clx,y}). 



?)(!-*) 



k=0 



On the other hand, the right hand side is 



£ 

\/j,\=n 



H^l - u;q,t] 



From [2H VI(6.11')] or [H (3.5.20)] we have H^l - u;q,t] = n[uB^(q, t)]' 1 . (See also Theo- 
rem H72] below.) Therefore this is equal to 

ch A G [n] 

V t n ~" J " r : =V(-l) d chg d (X' n ', A ,C?W) 



\fi\=n 
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thanks to fll . 13j) . Again a stronger geometric result is known (see (27J 5.2.1]) 

Hd( X M tf M) = is- k C[x,y}®f\ k C[x,y] if d = 0, 
1 otherwise. 

The module f\ k C[x, y] is infinite dimensional as a vector space. We can remedy it by 
replacing by £/[0] as follows. Let us take the simplest case k = 1 as an example. As 
E/[0] = O x[n] - (1 - g)(l - t)0 [n \ we have 

Rr xW (£/[0]) = S n C[x,y] - S^Cfoj/] 

from the above consideration, where we have used 

(l-q)(l-t)C[x,y]^C. 

This is nicely packed into generating functions: 

oo oo 

^^chRr xW (£/[0]) = (1 -u)^u"chRr xW (O xW ) = (1-«)Q[ * ' ]. 

n=0 n=0 ^ ^ ' 

Thus we get a polynomial 1 — u if we divide the answer by ^[tjz^jz^j]- This is an example of 
more general phenomena, which we will see later in Theorem 14.31 and Corollary 14.51 



2. Specialization 

The original Chern-Simons gauge theory depends on two integers A" and k, namely the rank 
of the gauge group SU (N) and the level k. In the corresponding quantum group approach, 
the Lie algebra is su(N) (or its complexification) and the quantum parameter q is taken as 
q = exp( 2ni /k+N). In the refined theory, another parameter t appears so that we identify q, t 
with parameters in Macdonald polynomials. But they are not arbitrary, and given by 

q = exp( 27ri /fe+/3A r ), t = exp( 27r */ 3 /fc+/3A r ), 

where /3 is a parameter for the refinement. The original Chern-Simons theory corresponds to 
P — 1, and it is sometimes better to think that /3 is an integer parameter so that q is a roots 
of unity. However we analytically continue f3 to the complex plane, as in [JJ. (A similar thing 
already appears in (2U VI§9].) Then we only impose the constrain 

(2.1) q k t N = 1. 

We study the behaviour of Macdonald polynomials specialized at (12. ip . 

In fact, this specialization will be performed in two steps. In the first step, we choose N. 
It corresponds to the projection from A to the ring Z[xi, . . . , xn] Sn of symmetric polynomials 
with finitely many variables. This is an elementary step. In the second step, we choose k and 
impose the condition (12.11) . In particular, we emphasize the role of Q[uBn(q, t)]" 1 appeared 
naturally in (11.131) . 

Note also that we break the symmetry between q, t and k, N here. We first choose t and A" 
to make a specialization. We can also choose q and k first. But the answer will be different. 
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2(i). Finite N. There is a natural inner product, suitable for Macdonald polynomials with a 
finite number of variables x = (xi, . . . , xjy). There are two versions, but we use ( , )jy [21], VI, 
(9.8)]. It is defined by 

(99 , , p p., fp p , tt l-r'W-n* c> x ( q ,t)n[t N -i q B x ( q ,t-i)] 

yi.i) {^x,^) N -ox^-rx,^x) q> tll 1 _ qal{s)+HN _ ll{s) _ 1 ~°^ Cx ^ n[t N B x (q,t^)} ' 

Note that the right hand side vanishes if there is a box s G A such that a'(s) = 0, l'(s) = N, 
i.e., /(A) > A". This is compatible with the property [211 VI (4.10)] that P x (x\, . . . , xn', q,t) = 
if /(A) > AT. In fact, this inner product has more natural definition on symmetric functions 
with N variables [21} VI §9]. So the radical of ( , )% is the kernel of the projection A — > 
Z[xi, . . . , xn] Sn ■ This inner product was used in [U Appendix A]. 
Let us recall a useful lemma, which will be used frequently later. 

Lemma 2.3 ([211 VI(4.17)]). Ifl{p) = N, we have 

P^xx, . . . ,x N ;q,t) = xx ■ ■ ■ £7viV(i,...,i)(xi, ...,x N ; q,t), 
where \x — (1, . . . , 1) denotes a partition (fix — 1, . . . , /xjv — 1)- 

2(ii). Specialization at q k t N = 1. We first specify what the specialization at q k t N = 1 means. 
A naive guess suggests to consider 2,[q ±l .t^}/ {q k t N = 1), but it has a trouble when k and 
are not coprime. We follow an idea in [6]. Let m = gcd(/c, N). We choose a primitive m th root 
of unity uj. We also choose Ui G C such that = u. We introduce a new variable v and 

set 

q = v , t = v~ k uii. 

For a Z[g ±:L ,t ^-module M, we consider M (8)z[ g ±,t±] C(w). This procedure is what we mean 
by the specialization at q k t N = 1. For example, we say / G M is nonzero at q k t N = 1 when 
/ ®z[g±,t±] is nonzero. In this case f~ l is well-defined in C(v). We say f~ l is well-defined 
at q k t N '= 1. 

Strictly speaking, we should call this the specialization at q k / m t N / m = uj. But we do not 
for brevity. Anyway u plays no special role except the property uj p = 1 if and only if p is a 
multiple of m in this paper. 

We consider 

We have the following simple observation. 

Lemma 2.4. (1) Q[t N B x (q, t" 1 )]" 1 vanishes unless /(A) < A 7 ". 

(2) Suppose /(A) < A". 4* = 1, tl[t N B x (q,t~ l )]- X vanishes if and only if /(A*) > k. 
Moreover the order of zero is 1 . 

Proof. (1) This is observed already above. 

(2) If /(A*) > k, we consider the box s in the first column and (k + l) th row. Then l'(s) = 0, 
a'(s) = k. Therefore t N ~ l ^q a ( s > = q k t N = 1. The converse and the second assertion can be 
proved as in the proof of Lemma 12.51 below. We leave them as exercises for the reader. □ 

Suppose /(A) < N. Then the corresponding Young diagram decomposes uniquely into the 
upper part consisting of rows shorter than A^ — 1 and the lower part of rows with exactly length 
N. Let us denote partition corresponding to the upper part by A. See Figure [TJ 
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Lemma 2.5. Suppose /(A) < N 

(1) c' x (q,t) is nonzero at q k t N = 1. 

(2) c\(q,t) is zero at q k t N = 1 if and only ifl(\ l ) > k and /(A ) < k. Moreover the order of 
zero is 1. 



A with l(X) < N 



A 



N ■ 



Figure 1. Two Young diagrams 



Proof. We prove only (2). The assertion (1) is easier to prove. 
Recall c x (q,t) = IL eA (l - q a ( s H l ^ +1 ). We have 

q a(s) t l(s)+l = w a(s) w Ar a ( s )_fc Ws)+ i)^ 

This is equal to 1 if and only if 

A(/( s ) + 1) = ^a(s), ^ (s) = 1. 
m m 

Since k /m and N /m are coprime, there exists an integer p such that l(s) + 1 = N p/m, a(s) = k vj m . 
Therefore = u p . Hence p must be a multiple of m. On the other hand, iV > Z(s) + 1 by 
our assumption /(A) < N. Therefore p < m. Combining two conditions, we must have p = 
or m. The first case cannot happen as l(s) + 1 = is not possible. In the second case we have 
l(s) = N — 1, a(s) = k. As /(A) < N, a box s with l(s) = N — 1 can occur only when /(A) = A^ 
and s is in the first column. Then a(s) = k means that s is in the (Ai — k) th row. Therefore we 
must have /(A*) = Ai > k. To have l(s) = N — 1, this row must have length N, which means 
that it is not contained in A. It is equivalent to /(A ) < k. Converse direction is clear. 

□ 

The above two vanishings are related as follows: 
Lemma 2.6. 

c x {q,t)tt[t N B^r 1 )} = c x (g,t)Q[t iV Bj(q,r% 
c'^tMt^qBx^r 1 )} = cL^tMt^qB^r 1 )}. 
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Proof. These assertions can be proved directly, but also follow from ( 13. 3 p below and Lemma 1231 
We have P A (t (Ar " 1) / 2 , . . . , t^^q, t) = P x (t (Ar " 1) / 2 , . . . , t {1 ~ N) / 2 ; q, t). And 2n(A) - 2n(A) = (Ax - 
Xi)N(N — 1) = (N — 1)(|A| — |A|). Hence we get the first equality. The second equality follows 
from 02.21) and the fact (P\, P\)% = (Pj, P\)%, which is a consequence of the definition in [211 
VI §9]. □ 



3. Refined Hopf link invariants 
3(i). S and T matrices. We fix N. Let us define the normalized S- matrix by 

(3.i) srr(g,t) d = f - q -nw»p x{t *-w . . . ,^-v 2 . q , t )p^ N -^q^\ . . .,e- N ^ N -, q,t). 

Here A and /i are partitions. We do not impose any restriction at this moment, but will do 
soon below. 

It is known that S x °f m (q,t) is symmetric under the exchange of A and fi. This is the 
Koornwinder-Macdonald reciprocity formula [211 VI(6.6)]. It is compatible with the expecta- 
tion that 5"° rm is the refind colored HOMFLY polynomial of a Hopf link, where colors are A 
and [i. We will see this later in Theorem 14.21 

Let us note that this vanishes unless /(A),/(/i) < N by [ZD, VI(4.10)]. Note also that 
Lemma 12.31 gives 

(3-2) Sr; m (q, t) = S%*(q, t), S^ m (q, t) = Sf™(q, t), 

where A, JL are as in Figured] as before. 

Let us rewrite S norm in terms of a modified Macdonald polynomial. The first term in (13.11) 
is given in [ZD, (6.11')]: 

1 _ n a'(s) f N-l'{s) 

.... t) = r-"-/-»w II i _V„, i „„ w ■ 

For example, the case A = □ gives 

t N/ 2 _ t -N/ 2 



Pait^ 2 , . . . , t (1 - N) / 2 ; q, t) = t N ^ 2 + ■■■ + t {1 ~ 



0/2 



f/2 _ t -l/2 



This is S£$ m (q, t), and is the refined Chern-Simons partition function for the unknot colored 
by the vector representation. 

The product of denominator is c\(q,t) in £jA] The numerator gives Q[t N B\(q, t -1 )] -1 by 
(II. ip . Therefore 

,-(jv-i)|A|/ 2 +n(A) 

^ P^ N ^\ . . . , t^; q, t) = J 

il[t N B x {q,t 1 )Jc A (g,i) 

Next we rewrite the second term using the plethystic substitution § |l(i) Note that 



,_(jv-i)/ a 
1 -t 



l-t N 



i-t 

i=i i=i 



(l-t^far 1 )) 
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Therefore 



Multiplying two factors, we get 
Proposition 3.4. 



+ n(X)+n{u)-{N-l)(\\\ + \n\)h tt r-i J.JV /( 7--1V „ 

r-norm/ -X _ |A| | M | /jv / J^jl - t A A (g,t J 

^' > q nJr, An (n +\ Cl\+N R. („ +-\\\ 



A " W ' 7 " c A (g,t)c M ( g) t) n[t»B x (q,t-i)} 

At this stage it is clear that this is well-defined element in Q>(q,t), as G Z[g ±1 ,t ±1 ] <g> A. 
Moreover it vanishes unless /(A), < ^ as we remarked before. 
Properties at the specialization at q k t N = 1 are as follows. 

Lemma 3.5. Assume Z(A),/(/x) < N. Let X, ~p as in FigureUi Then S^° im (q,t) is well-defined 
at q h t N = 1. Moreover, it vanishes at q k t N = 1 unless /(A*), /(/I*) < k. 

Proof. By the symmetry A <H- it is enough to study c A (g, t^ft^P^g, 
Since we have S%° rm (q,t) = S^ m {q,t) by (Q, we may assume /(A) < AT. 

By Lemma EH Q[t N B x (q, t^ 1 )]" 1 vanishes unless /(A*) < jfe. Its zero may cancel with the 
zero of c\(q, t), but this cannot happen by Lemma [231 (2). as we have assumed A = A. Therefore 
the both assertions follow. □ 

We give the duality between q t. It is related to the level-rank duality for the affme Lie 
algebra. This will not be used later. 

Proposition 3.6. Suppose q k t N = 1 and consider S™™(t~ l ^q' 1 ) defined by replacing k and 
N. Then 



^° rm (?,*) / fq 



|A|+H 

S n ?™(t Q ) 

(P\> P\)< ? ,t(P At , P/i)?,* 
Proof. The left hand side is 

onQrm/y-1 -In = .|A|N/fc g fV I 1 ~ g A* >_gM » gj 

AVl ' g J c A *(t-i, g -i)v(t-i,g-i) " n[ g -*S A i(t-i,g)] 

By (jl.5p and some identities in §jAj we get 

^[l-g-^^r 1 ,?)^- 1 ,?] _ H^l-^Axfat- 1 );^- 1 ] 



n[q- k B xt (t-\q)} Q[t N B x (q,t^)] 

and 

^-n(A*)-n(/i t )+(fc-l)(|A t ! + |n t |)/2 g-7i(A')-n(>* )-(A:-1)(I**I+!m*!)/2 ^_ j \ | X\ + \ fJ ,\^n(X)+n{fi) ^(fc+l)(l>* I + Im* D/2 

CAtCt- 1 ^- 1 )^^- 1 ,?- 1 ) = ^(g- 1 ,*- 1 )^^- 1 ,*" 1 ) = c A (g,t)c^(g,t) ' 

The assertion follows from (P\,P M ) 9 ,i = S\^ c 'x(<i^)/c x (q,t). □ 
We now introduce the normalized T-matrix as 

T™™(q,t) = ^ g l|A|| 2 /2-|A|V^-||A*|! 2 /2 + V|A|/ 2> 

Note that we have 

(3.7) T A 7 m (g, t) = T A 7 m (g, t), T^ m (q, t) = T^(q, t). 
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In [T], 5*°° rm (g, t) appears as 

S%° im {q, t) = (Px{xi, ...,x N ;q, t) \-^-\P^x u ...,x N ;q, t))%, 

^00 

where S is the S'-matrix, and S<dq is a certain explicit function, which can be identified with 
refined Chern-Simons partition function for S 3 . Then the linear operator corresponding to 
s /s m is given by 



s x Mt) 



S^T(q, t) _ rW+"^- (Af - 1)(|A|+l ^/ 2 jyi - t N A x (q, r 1 ); q, t" 1 ] 
(Px,Px)'^ ~ c' x {q,t) Cll {q,t) Q[t N ^qB x (q,t^)] 

d x {q,t)c^q,t) fit^fa, t" 1 )] 

where the last expression follows from S'"° rm (q', t) = S^ m (q, t). When we compute a multi- 
plication of S novm , we must use S x (q,t). More precisely we consider (S x ) as a matrix whose 
indices are partitions A, \i with l(X), < N — 1 and /(A*), £(//) < k. The same is true for 

ynorm 

We define 

d f v S' norm (g i) 

(3.8) Z(x,y;q,t) = J2 tp vv> m 5~vT P a(^i, ■ ■ ■ t x N ; q,t)P^(y u . . . ,y N ; q,t), 

^ \P\, P\) n\P^ Py-I n 

where the summation is over partitions with /(A), < N — 1. We have 

(^,2/;9,i),/(?/)}';=(f/) (*), 

\^00 / 

where the inner product is taken for the variable y. This means that Z(x,y; q,t) is the kernel 
function of the operator s /s m . 

Remarks 3.9. (1) Note that S x Jq,t) is always well-defined at q k t N = 1 if /(A) < N — 1, and we 
have S^q, t) \ q k t N- t = if ^(m*) > k. However S^(q,t)\ may not vanish even if /(A*) > k. 

This does not cause any trouble, as we eventually compute (0|a monomial in S, T, 7V".|0). The 
answer remains the same whether imposing /(A 4 ), /(//) < k or not. This observation says that 
the refined Chern-Simons partition function is a rational function in q, t and t for a fixed N. 

Similarly the sum for the kernel function Z(x,y; q,t) in (I3.8P is infinite for generic q, k, but 
is a rational function in q, t if we take a part with given degrees in x and y. It gives the 
operator s /s m at q k t N = 1. 

(2) Let us also consider what happens when we consider N (or t ) as a variable. Note that 
S^iiQit) = if Z(/i) > N. This is obvious. For the case /(A) > iV, we use the second or last 
expression for the above /^(g, £) to see that it is well-defined even if /(A) > N, though it may 
not be 0. Thus the situation is similar to (1) above. 

However there is a difference: we impose the condition l(X), l(/J,) < N — 1, rather than < N. 
In fact, when we only assume /(A), Z(/x) < N, we cannot multiply the matrices (S x ), as 
infinitely many entries are nonzero. As entries with /(A) or /(/i) = N can be determined by 
entries with < N thanks to relations (13.21 13 .7p . the author hope that this is not a serious issue, 
even though he does not know a resolution at this moment. 
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3(ii). A stable version. Replacing t N by u, we define a stable version of S norm : 

V c A (g,t)c,(g,t) n[«B A (g,f-i)] 

Another variable v will be identified with q l / N at the specialization q k t N = 1 as in S2(ii) , but is 
an independent variable at this moment. It is expected that q l l N appears only as the framing 
factor p. 

For example, we have 

W V2 _ „-l/2 

S^(q,t,u,v) 



t l/2 _ £-1/2 • 

We consider a stable version of the kernel function in (13.81) : 

S?°™(q,t,u,v) 



\,fj, 



(P\, P\)q,t(Pti, Pfi)q,i 



We do not impose any constrain for the range of A, \x. So this should be considered as an 
element in the completion with respect to degrees in x, y. 

Note that this does not give the finite kernel function (13.81) at u = t , q k t N = 1. First because 
the inner product is different. Second because the summation is not over /(A), /(//) < N — 1. 
It is over < N. 

The first difference can be resolved by modifying the kernel function as 

(o im v gggi^ niuBx&t- 1 )] n[uB,(q,t-^)] 

But the author does not know how to solve the second difference as mentioned in Remarks l3.9( 2). 
We change the inner product from ( , to ( , )* by (11.91) to get 

(3.11) 



/(*) 



{P\, P\)q,t(P[i, Pfl) 



q,t 



g(y) 



q,t 



:-ty 



deg/-degg 



/ [ 



A' 



1-t 



-l 



R\ VP P\ A ^-t J 



A,// 



Y 



1 - t 



-1 



by (11.91) . Here the inner products are taken for both x and y simultaneously. We have 

\- S%r(q,t,u,v) X Y 

2^ /p pn /p p\ p a[- — : ; q, t]P„[- — -; q, t] 



A.// 



{P\, P\)q,t(Pfi, Pfj) 



q.t 



1-t' 



1-t' 



A, /Li 



S^ m (q,t : u,v)t n ^ +n ^ ~ - _ 

/ / . w / H\{x;q,t )H^y;q,t ). 



In order to match with computations in ^l(iii) , we replace t by t . Then the above becomes 



A, /i 



H x {x]q,t)H pL (y-q,t). 
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We absorb the term (-t)~ deg/ ~ degs | t ^ t _ 1 in ( 13TTT) into the above as (-t)l A l+H We then 
substitute Proposition 13.41 to get 



; ,-|A|M (tM) -.i^i.i./» g| ,[l- M A A fa,t) W ,t] ^ - 

(3 ' 12) t<^.<;w ^'''O^'^' 

where we have used ( 11.41 II . 10)) . 

4. Geometric interpretations via Hilbert schemes 

We give geometric interpretations of the stable version of the kernel function ( 13. 11)) or ( 13 .12)) 
in the previous section via Hilbert schemes of points on the plane. We give two versions, a 
naive one and the second one. The second one looks better though it contains an infinite sum. 

Let us first replace the inner product ( , )* in ( 13. lip by ( , ) in (11.19)) . where we suppress 
the functor <3>. It corresponds to applying operators V x , V y to H x (x; q, t), H^y; q, t) in (I3.12p . 
Here V x means the V-operator for the variable x and similarly for V y . We denote the result 
by Z(x,y;q,t,u,v), i.e., 

Z{x,y;q,t,u,v) = > — = = ~ ' r ——. ^ -V H x (x]q,t)\/ y HJy:q,t). 

Therefore 

(4-1) AMU- = (f [j^l Z(x,y;q,t,u,v) <?[y^ ]J ■ 

Recall that f[ x /i-t] and g[ Y /i-t] are considered as classes in Kq* x c*(Lx ) under $ (ll.2ip . 
The expression Z(x, y\ q, t, u, v) in the middle is a generating function of classes in the localized 
equivariant if-group Kc* x c*(X^ x X^)®z{ g ±i j±i]®Q(q, t). As the inner product ( , ) is simply 
given by the functor HT X [n] (— Cg> L — ), the above (14.11) is simply given by the convolution product 

X Y 
BT xln]xxM (pJ(/[— ]) ® L Z{x, y; q, t, u, v) ® L p*( y [_])). 

Recall that we have a modified version of Z(x, y; q, t, u, v) in (13.101) . where we have an extra 
factor 

n[uB x (q,t)] n[uBn(q,t)] 
n[uqtB x (q,t)]Q[uqtB^q,t)X 

(Recall that we replaced t by t" 1 .) This factor appears as AjH x (x; q,t)A V cH fl (y; q,t) with 

f ^ = n[uqtX] ' 
Under $, Af gives an operator multiplying 

A- M g*o [wl 

We do not understand whether Z or its modified version is correct yet, but we only treat 
Z(x, y; q, t, u, v) as the difference is just given above. 
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4(i). A naive geometric interpretation. First note that 

V x H x (x;q,t) I x 



by (11.12)) and that H\ corresponds to the class I\ for the fixed point as we explained in £l(iv) 
The factor 1 /a[uB x {q,t)} is the character ch f\_ u Oj of the exterior power of the tautological 

bundle at the point I\ by (j!.13p . 

As is a linear combination if M = Yl v -^m(0S t) s v °f Schur functions, we have 

H^[l-uA x] q,t] = Y,K^{q,t)s u [l-uA x }. 

Since A\(q,t) is the character ch£( ,/ A ) by fU.15p . we can interpret s u [l — uA\] as the fiber at 
I\ of s u [0 — uS/[0}}, the Schur functor s u applied to O — u£/[0}. Moreover K„^(q,t) is given 
by the Procesi bundle V: 

K^(q,t) = ch (P„) v 
Therefore Z(x, y; q, t, u, v) is the sum of the fiber of 

Q){K_ u O^®s v [O-u£/[0]])^V u 

V 

at (Ix,Ift), divided by 

ch /\^T* x X [lxn ch /\_ 1 T; M X [l ' i|] . 

This is the first geometric interpretation of Z(x,y; q,t,u,v). It is a compact form, but seems 
to difficult to its geometric meaning. 

4(ii). The second geometric interpretation. In [TJ The proof of Th. 3.3] the symmetry of 
i s proved by an application of the following: 

Theorem 4.2. 

H^l -uA x (q,t);q,t] -u / . /- -XA^q, t) \ ~ -uXA x (q,t) 

n[uB x (q,t)} L (l -t)(l -g) J \ V [ (l-g)(l-t)V ' L (l-g)(l-t) 

Since V is self-adjoint with respect to ( , )* from fll.lOp . or its geometric interpretation in 
£ |l(iv) , the symmetry under A -H- /i is clear. 

It should be also remarked that this is an equality in Q(g, The left hand side is in 

1i[q , t^ju], but the right hand side contains an infinite sum in powers of u. 

Note that there is a typo in [7j. The second u was missing. 

Let us understand the right hand side in terms of Hilbert schemes. Let us first remark that 

_ / oo \ - 1 



which we calculated as 

chRr xW (C xH 



K n=0 



in ( Tl~23l) . 

The second term can be also calculated to get 
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Theorem 4.3. 

if M [l-«A A (g,t);g,t] 

n=0 



Q[uB x (q,t)} ^ U chRT x M (O xM ) 

(4-4) 



n=0 



Here (uV 1 i?A)[^/[0]; g, t] is the plethystic substitution of the universal sheaf £/[0] at the 
fiber x X' n ' to (co'V~ 1 H x )(x; q, t). A little more concretely, as 

wV _1 ff A (s; g, = #a0; g" 1 , r 1 ) = ^ K CTA (g-\ r 1 )^, 

(7 

it is nothing but 

(7 

where s CT [£/[0]] is the Schur functor applied to £/[0], as in the previous subsection. 

Before giving the proof of Theorem 14. 3[ let us consider a special case A = 0. We can slightly 
change the expression. 

Corollary 4.5. 

oo oo 

(Vs A *)[l -u}Y,u n chRT xln] (O xln] ) = Y,u n chRT xln] (s x [S/[0}]). 

n=0 n=0 

This special case is already very surprising formula as it computes holomorphic Euler char- 
acteristics of universal sheaves twisted by arbitrary Schur functors. Note that V is defined in 
terms of H^, and hence is very complicated to compute Vs A *. 

Proof. Theorem 14.31 with A = gives 

oo 

(4.6) H,[l - u; q, t] = ^[ {1 _~ ) ^_ q) ] J2 n " ch ^^((uV^H.) [S/[0]; q, t}). 
If we write 

uV~ 1 H ll (x-q,t) = Y,K^{q-\r 1 )s Xl 

A 

we have 

A 

Since the matrix t -1 )) is invertible, we have the assertion. □ 

Let us start the proof of Theorem 14.31 We first prepare the following. 
Lemma 4.7. 

(-1)^H,[-A x (q,t)}q n ^k n ^ = (-l)MH x [-Ap(q,t)]q^fW>. 



Proof. Let us consider the left hand side of (I4.4p . where the denominator is replaced by the 
middle term in fll . 13j) . Since it is symmetric under the A, /i, we have 

H,\l - uA x (q, t); q, t] H(l - uq a 'W^) = H x [l - uA^(q, f); q, t] - uq a 'W^). 

sex sei-i 
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Therefore 

- A x (q,t); q,t] ^n 1 - = H^u' 1 - A^t); q,t] JJ^" 1 - g W^). 



sex 



Setting u 1 = 0, we get the assertion. 



□ 



Now we are ready to calculate the second term in the right hand side of Theorem 14.21 By 
(11.611. EID we have " 



(l-g)(l-t) J ' L (l-g)(l-t) 



E 



o*(g-VK(g, t- 1 ) c v (q-\t)d v (q, t- 1 ) 

q-M^t-WH^-A^q, t)]H u [-uA x (q, t)} 



Cviq^^ty^t- 1 ) 
By Lemma [4.71 this is equal to 

(_l)|A|+|Ml g -n(A«)-n(^) r n(A)-n( M ) tt |^|ff A ^_^^ ) ^]^^_^( g) ^] 

We have 

( _l)|A| g -n(A0 t -n(A)^ A[ _^ (g)t)] = H X )[A v {q,t)]. 

Note that A,(g, t) is the character of £\oxi„, and the denominator is /\_ 1 T/ j/ X[ n l by (11.121) . 
Hence the Atiyah-Bott-Lefschetz fixed point formula gives the right hand side of Theorem 14.31 
This completes the proof of Theorem 14.31 

We substitute Theorem 14.31 to the definition of Z(x, y\ q, t, u, v) to get 

Z(x,y;q,t,u,v) 

^ v~\ x \M(tv)- (W+M) / 2 
= E lH h uh j. ^^'W"-') 

ZZo un ch Rr A-w [£/[0] ; <z, t] ® (wV" 1 ^) [5/[0]; g, *] 



By fTPD 



V a ff A (x; g, ty(V^)- 1 g A (^; g, t) = ^ # A (x; g, t)w/f A (*; g, t) 



A 



(if A , H\), 



(H\, H\)* 



XZ 



L(i-g)(i-t) 



ft 



(1 -?)(!-*) 



Using the usual Cauchy formula for Schur functions, we write this as 

A 

Since l /i-q = ch.C[y], £/[0]/(l — q) means the slant product £/{x = 0}. Similarly £/[0]/(l — t) 
means £/{y = 0}. 

Therefore we get our main result. 
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Theorem 4.8. 

X , r Y 



(4.9) Z(x,y;q,t,u,v) = ^ v~ WM (tu 



-(|A| + |Ml)/2 



1 - t J M T - q 
v En=o" w ^RTxwK[g/{g = 0}] ® s M [£/{y = 0}]) 

X 



Er=o M ™ ch Rr xw (^[»] ) 

Note that s A [X/(l - *)], s M [F/(l - q)\ give bases for ©^xc-l^ 1 ). ©^c*xc*(^ n] ) re- 
spectively by j jl(v) Therefore the above is a class in K c * xC *(L^ x 1$). 



Let us give several further remarks on this expression. 

Remarks 4.10. (1) The second term in the right hand side is a polynomial in u as manifest in 
Theorem 14.21 though it is a priori a formal power series in u. Then ( 14. 9 p is an equality in a 
formal power series in vT 1 . Otherwise it contains infinitely many both negative and positive 
powers of u, and hence is not well-defined. 

(2) The class s\[£/{y = 0}] is not integral (even for A = □). But the denominator is 
somewhat controllable as follows. Let us rewrite the above as 

Z(x,y;q,t,u,v) = £ t,-^^-^ r ^ Y _ 1 



(l-t)(l-g) J ^(l-t)(l-g) J 
Er=o^chRr xW ( gA [g/[0]] ® a M [g/[0]]) 

£r=o^ chRr xw(OxM) 



By Corollary 14.51 this is equal to 

V? I 

(l-t)(l-g) J ^(l-t)(l-q) 



where iV^ is the (ordinary) LittlewoodRichardson coefficient. We write 
s A (x) = £)X Al/ (t)s I/ [(l -t)X] = J2XxAt)S u (x;t), 

V V 

where S v {x] t) is as in [211 HI§4]. Replacing X by X/ (1 — i)(l — q) we get 

^ (i-fKi- g ) 1 = ^^ M i^ ] - 

This means that though Z is not an integral class in Kc*xC*(L^ x L^), the denominator 
only comes from X\ v {i) and X^ a (q). The transition matrix X\ v {t) is given in [2TJ III§6], and 
the denominator is of the form 

i(A) 

8=1 

where A = (l m i2 m2 ■ • • ) and (p m (t) = (1 - t)(l - t 2 ) ■ ■ ■ (1 - t m ). 

Appendix A. Partitions 
We collect some notation on partitions. 

A partition A is a nondecreasing sequence A = (Ai > A2 > A3 > ■ ■ • ) of nonnegative integers 
such that A at = for N ^> 0. We usually do not write the entry Aj if it is zero. We identify 
A with a Young diagram Y\ as in [23]. The transpose of A is denoted by A*. We consider Y\ 
as a subset of boxes s in the xy-plane. For a box s in the xy-plane and a Young diagram Y, 
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we denote by ay(s), Zy(s), a'(s), l'(s) the arm-length, leg-length, arm-colength, leg-colength 
respectively. The hook length is defined by hy{s) = a Y (s) + /y(s) + 1. We also write it a\(s), 
h( s ), h\(s) if Y corresponds to a partition A. (We shall use only deal with rank 1 case, i.e., 
the case of Hilbert schemes of points.) When a box s is in the Young diagram, we denote by 
s G Y\, or simply by s G A. Since the diagram is clear from the context (because we only 
consider 'rank 1' case), we simply denote them by a(s), l(s), h(s). 

Unfortunately our convention of arm/leg-(co)length functions is different from [161 [10]. Our 
Young diagram is rotated by 90° anticlockwise. 

We give some standard notations and useful formulas: 

|A| = A, = #{s € A} = the number of boxes in Y\, 



l 

"(*) = £(«'- 


i)a, = J2 /7 ( s ) = E 1 


i 




n(A') = a '( 


s) = ^ay(s), 






k(X) = 2(n(X t ) 


-n(A)), 


l|A|| 2 = 5> 2 = 

■i 


= 2n(A*) + |A|, 


B x (q,t) = J2<l a ' {s) t l ' {s) =BAt,q), 


seA 




A x (q,t) = l-( 


[l-t)(l-q)B x (q,t) = . 


c x (q,t) = l[(l 




seA 




^(?,*) = II( 1 


_ q a(s)+l t l(s)j = Cxt {t A ) 



seA 



Appendix B. Symmetric polynomials 



Our notation for symmetric polynomials is standard, and follows |21j . We denote by e&, 
hk-, Pk the k th elementary, complete, and power sum symmetric functions. Schur functions 
corresponding to a partition A is denoted by s x . 

The ring of symmetric functions limZ[xi, . . . ,Xn] Sn is denoted by A. The Schur functions 
s\ define an integral base of A. 

An inner product on A is defined by 

(s\, Sf,} = 5 X fi. 

The ring A is isomorphic to the direct sum @ n R(S n ) of Grothendieck group of representa- 
tions of the symmetric group S n of n letters. Here the multiplication is given by 

v ■ w = indf™+ n s (vmw). 

See [211 I§7] . A Schur function s\ corresponds to an irreducible character. The isomorphism 
is called the Frobenius characteristic map. 

The degree n subspace of A is denoted by A n . It is isomorphic to R(S n ). 
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